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The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 

after SIX (6) MONTHS from the mailing date of this communication. ^ , 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

Responsive to communication(s) filed on . 

2a)n This action is FINAL. 2b)l3 This action is non-final. 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) 13 Claim(s) 1-20 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) 13 Claim(s) 1-20 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 3 The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)n accepted or bO objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held In abeyance. See 37 CFR 1 .85(a). 
1 !)□ The proposed drawing correction filed on is: a)^ approved b)^ disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

1 3) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 

a)nAII b)n Some* 0)0 None of: 

1 .□ Certified copies of the priority documents have been received. 
2.n Certified copies of the priority documents have been received in Application No. 



3.n Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

1 4) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 1 9(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 
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Application/Control Number: 09/822,923 
Art Unit: 2672 

DETAILED ACTION 

Specification 

1. The disclosure is objected to because of the following informalities: On page 13, line 18, 
"display widow 200" should be "display window 200". On page 15, line 18, "claibration" should 
be "calibration". On page 16, line 3, "has been shifted". Appropriate correction of all mistakes is 
required. 

2. The applicant or their representatives are urged to review the specification and submit 
corrections for all mistakes of a grammatical, clerical, or typographical nature. 

Qaim Objections 

3. Claim 5 is objected to because of the following informalities: On page 31, line 2 of claim 
5, "standard deviation of the pixels" should be "standard deviations of a hue of the pixels". 
Appropriate correction is required. 

4. Claims 6 and 8 are objected to because of the following informalities: On line 1 of 
claims 6 and 8, "are" should be "is". Appropriate correction is required. 

5. Claim 11 is objected to because ofthe following informalities: On hne 3 of claim 11, 
"are greater a predetermined amount" should be "is greater than a predetermined amount". 
Appropriate correction is required. 

6. Claim 12 is objected to because ofthe following informalities: On line 3 of claim 12, 
"pixels in the image frame are to be a part of should be "pixels in the image firame are to be part 
of. Appropriate correction is required. 



8. 



"are" 



Page 3 

Application/Control Number; 09/822,923 
Art Unit: 2672 

7. Claim 13 is objected to because of the following informalities: On line 1 of claim 13, 
"The method of claim 13". It is assumed that claim 13 is dependent on claim 9 and therefore it 
should be changed to "The method of claim 9". Appropriate correction is required. 

Claim 15 is objected to because of the following informalities: On line 2 of claim 15, 
should be "is". Appropriate correction is required. 

9. Claim 20 is objected to because it is the same as claim 14. 

Claim Rejections - 35 USC § 112 

10. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

-me speaficatxon shall conclude w:th one or more clakns particularly pomtmg out and distinctly clanmng the 
subject matter which the applicant regards as his invention. 

1 1 Claim 4 is rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for failing 
to particularly point out and distinctly claim the subject matter which applicant regards as the 



invention. 

Claim 4 recites the hmitation "the HSV thresholding device" in line 2 of the claim. There 
is insufficient antecedent basis for this hmitation in the claim or its dependent claim. 

Claim Rejections - 35 USC § 102 
12. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

treaty in the Enghsh language; or 
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(2^ a patent granted on an application for patent by another filed ,n the United States before the invention by 
SpLaX S nt except that a patent shall not be deemed filed m the United States for the purposes of this 
:£ecSn bafei on the fLg of an international application filed under the treaty defined m section 351(a). 

13. Claims 1-20 are rejected under 35 U.S.C. 102(e) as being anticipated by Bradski et al. 
U.S.Pat. No. 6,363,160. 

14. Claim 1: 

The Bradski reference teaches an automated calibration system (figure 1) for tracking a 
colored object through a series of frames of data (column 3, lines 34-55), comprising ; 

(a) A first processing device (i.e., the computer system 157) to execute a program (figure 
1), wherein the program displays at least one image frame from an image input device (i.e., a 

camera. See column 4, lines 1-10); 

(b) An image selection device to select the colored object in the at least one image fi-ame 
(column 3, lines 59-67, column 4, lines 1-10, and column 7, Unes 33-54); 

(c) An image source device (video source 1 102 of figure 1 1) to provide a hue saturation 
value (HSV) data array of pixels in the colored object (column 13, Unes 45-52); 

(d) A second processing device (i.e., the computer system 157 of figure 1 or the micro- 
processor based device 1 106 of figure 1 1) to analyze the HSV data array, wherein the second 
processing device determines characteristics of the pixels in the at least one image fi-ame that are 
to be associated with the object based on a probability (figures 4-5, and column 4, lines 1-57). 

Claim 2: 

The claim 2 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of HSV threshold device. However, the Bradski reference further discloses the 
claimed limitation of the HSV threshold device (column 4, lines 1-34). 

Claim 3: 
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The claim 3 eacompasses .he s^e scope of invenuoa as « of Cairn 1 except ada-ona, 
claimed l«atio„ of a digital camera. However, the Bradski reference firf,er discloses the 
claimed limitation of the digital camera (column 4, lines 1-57). 

Claim 4; 

The claim 4 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of a s.ngle device. However, the Bradski reference further dtsdoses the 
claimed limitation of the digital camera (figure 1). 

Claim 5: 

The claim 5 encompasses the same scope of mvention as that of claim 1 except additional 
cWmed Ihnitation of the second processing device calculating a mean hue and a standard 
deviation of a hue of the pixels representing the colored object. However, the Bradski reference 
fiirther discloses the claimed limitation of the second processing device calculating a mean hue 
and a standard deviation of a hue of the pixels representing the colored object (column 4, lines 2- 
57, column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 22-44). 
Claim 6: 

The claim 6 encompasses the same scope of invention as that of claim 5 except additional 
claimed lunitation of the colored object not being tracked if the mean hue or the standard 
deviation of the hue are less than predetermined levels. However, the Bradski reference ftirther 
discloses the claimed limitation of the colored object not being tracked if the mean hue or the 
standard deviation of the hue are less than predetermined levels (column 4, lines 2-57, column 
10, lines 1-16, column 5, lines 31-63, and column 6, lines 22-44). 
Claim 7: 



Application/Control Number: 09/822,923 ^^8^ ^ 

Art Unit: 2672 

The claim 7 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of the second processing device calculating a mean saturation and a standard 
deviation of a saturation of the pixels representing the colored object. However, the Bradski 
reference further discloses the claimed limitation of the second processing device calculating a 
mean saturation and a standard deviation of a saturation of the pixels representing the colored 
object (column 4, lines 2-57, column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 
22-44). 

Claim 8: 

The claim 8 encompasses the same scope of invention as that of claim 7 except additional 
claimed limitation of the colored object not being tracked if the mean saturation or the standard 
deviation of the saturation is less than predetermined levels. However, the Bradski reference 
further discloses the claimed limitation of the colored object not being tracked if the mean 
saturation or the standard deviation of the saturation is less than predetermined levels (column 4, 
lines 2-57, column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 22-44). 
15. Claim 9: 

The Bradski reference teaches a method of calibrating a computer-vision system (figure 
1) to track a colored object through a series of frames of data (column 3, lines 34-55), 
comprising : 

(a) Executing a program a program (figure 1) to display at least one image frame from an 
image input device (i.e., a camera. See column 4, lines 1-10); 

(b) Providing a calibration selection device to select the colored object in the at least one 
image frame (column 3, lines 59-67, column 4, Unes 1-10, and column 7, Unes 33-54); 
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(c) Performing calibration processing to ensure the object selected is trackable throughout 
the series of frames (video source 1102 of figure 1 1), wherein the calibration processing analyzes 
pixel data of the object (column 13, lines 45-52); and 

(d) Creating a table from the pixel data for the object (figure 4). 
Claim 10: 

The claim 10 encompasses the same scope of invention as that of claim 9 except additional 
claimed limitation of converting a pixel data array for the at least one image frame from a red- 
green-blue colorspace (RGB) data array to a hue- saturation-value colorspace (HSV) data array. 
However, the Bradski reference fiirther discloses the claimed limitation of converting a pixel 
data array for the at least one image frame from a red-green-blue colorspace (RGB) data array to 
a hue-saturation-value colorspace (HSV) data array (column 4, lines 1-57). 
Claim 11: 

The claim 1 1 encompasses the same scope of invention as that of claim 9 except 
additional claimed hmitation of applying the pixel data from an entire frame to the pixel- 
classification look-up map wherein if the amount of the pixels associated with the object is 
greater than a predetermined amount, the calibration method restarts. However, the Bradski 
reference fiirther discloses the claimed hmitation of applying the pixel data from an entire frame 
to the pixel-classification look-up map wherein if the amount of the pixels associated with the 
object is greater than a predetermined amount, the calibration method restarts (figure 4). 

Claim 12: 

The claim 12 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of determining which of the pixels in the image frame being part of 
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the colored object selected based on a probability. However, the Bradski reference further 
discloses the claimed limitation of determining which of the pixels in the image frame being part 
of the colored object selected based on a probability (figures 4-5). 
Claim 13: 

The claim 13 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of thresholding the HSV data array of pixels in the colored object. 
However, the Bradski reference further discloses the claimed limitation of thresholding the HSV 
data array of pixels in the colored object (column 4, lines 1-57). 

Claim 14: 

The claim 14 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of calculating a mean hue and a standard deviation of a hue of the 
pixels in the colored object. However, the Bradski reference further discloses the claimed 
limitation of calculating a mean hue and a standard deviation of a hue of the pixels in the colored 
object (column 4, lines 2-57, column 10, lines 1-16, column 5, lines 3 1-63, and column 6, lines 
22-44). 

Claim 15: 

The claim 15 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of restarting the calibration method if the mean hue or the standard 
deviation of the hue is less than predetermined levels. However, the Bradski reference further 
discloses the claimed limitation of restarting the calibration method if the mean hue or the 
standard deviation of the hue is less than predetermined levels (column 4, lines 2-57, column 10, 
lines 1-16, column 5, lines 31-63, and column 6, lines 22-44). 
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Claim 16: 

The claim 16 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of calculating a mean saturation and a standard deviation of a 
saturation of the pixels representing the colored obje^. However, the Bradsk, reference toher 
discloses the clatmed limttatton of calculating a mean saturation and a standard deviation of a 
saturation of the pixels representing tite colored object (column 4. hues 2-57. column 10, hues 1- 
16, column 5, lines 31-63, and column 6, lines 22-44), 
Claim 17: 

The claim 17 encompasses the same scope of invention as tttat of claim 16 except 
additional claimed Umttation of restarting the calibration tnethod ,f the mean satitration or the 
standard deviation of the saturation is less than predetermined levels. However, tite Bradski 
reference further discloses the claimed limttation of restarting the calibration method if tite mean 
saturation or the standard deviation of tire saturation is less than predetermined levels (column 4. 
lines 2-57. column 10, lines 1-16. column 5, lines 31-63, and column 6. lines 22-44). 
Claim 18: 

The claim 18 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of allowing the user to select the colored object. However, the 
Bradski reference teher discloses the clatmed Umttation of allowing tite user to selea the 
colored object (column 4, lines 1-57). 
Claim 19: 

The claim 19 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of a pixel classification look-up map. However, the Bradski 
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.eferencefUrtherd.closestheclain.edUn..atio^ 
5). 

Claim 20: 

The cla,m 20 encompasses ,he same scope of mvention as that of claim The claim is 

subject to the same reason given in claim 14. 

Conclusion 

,6. The pnor an made of record and no. relied upon is considered pertinent to apphcanfs 

disclosure. 

a. Darrel et al. U.S. Pat. No. 6,445,810 discloses techniques from computer vision 

and computer graphics to robustly Brack a target. 

b. Matey et al. U.S. Pat. No. 5,649,021 discloses a color detector processing the 
.mager signals to detect a range of colors in the image represented by the imager signals. 
,7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed ,0 Jin-Cheng Wang whose telephone number ,s (703) 605-1213. 
The examiner can normally be reached on 8:00 AM - 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessM, the examiner's 
supervisor. Mrke R^av, can be reached on (703) 305-4713. The fax phone numbers for the 
organr^ation where this applicatron or proceeding is assigned are (703) 308-6606 for regular 
communications and (703) 308-6606 for After Final communications. 

Any inquiry of a general nature or relatmg to the status of this applicatron or proceedmg 
should be directed to the receptionist whose telephone number is (703) 395-3900. 
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